{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Caching\n",
    "\n",
    "- Author: [Joseph](https://github.com/XaviereKU)\n",
    "- Peer Review : [Teddy Lee](https://github.com/teddylee777), [BAEM1N](https://github.com/BAEM1N)\n",
    "- Proofread : [Two-Jay](https://github.com/Two-Jay)\n",
    "- This is a part of [LangChain Open Tutorial](https://github.com/LangChain-OpenTutorial/LangChain-OpenTutorial)\n",
    "\n",
    "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/LangChain-OpenTutorial/LangChain-OpenTutorial/blob/main/04-Model/03-Cache.ipynb)[![Open in GitHub](https://img.shields.io/badge/Open%20in%20GitHub-181717?style=flat-square&logo=github&logoColor=white)](https://github.com/LangChain-OpenTutorial/LangChain-OpenTutorial/blob/main/04-Model/03-Cache.ipynb)\n",
    "## Overview\n",
    "\n",
    "```LangChain``` provides optional caching layer for LLMs.\n",
    "\n",
    "This is useful for two reasons:\n",
    "- When requesting the same completions multiple times, it can **reduce the number of API calls** to the LLM provider and thus save costs.\n",
    "- By **reduing the number of API calls** to the LLM provider, it can **improve the running time of the application.**\n",
    "\n",
    "In this tutorial, we will use ```gpt-4o-mini``` OpenAI API and utilize two kinds of cache, ```InMemoryCache``` and ```SQLiteCache```.  \n",
    "At end of each section we will compare wall times between before and after caching.\n",
    "\n",
    "### Table of Contents\n",
    "\n",
    "- [Overview](#overview)\n",
    "- [Environement Setup](#environment-setup)\n",
    "- [InMemoryCache](#inmemorycache)\n",
    "- [SQliteCache](#sqlitecache)\n",
    "\n",
    "### References\n",
    "- [SQLIteCache](https://python.langchain.com/api_reference/community/cache/langchain_community.cache.SQLiteCache.html#langchain_community.cache.SQLiteCache)\n",
    "- [InMemoryCache](https://python.langchain.com/api_reference/core/caches/langchain_core.caches.InMemoryCache.html)\n",
    "----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Environment Setup\n",
    "\n",
    "Set up the environment. You may refer to [Environment Setup](https://wikidocs.net/257836) for more details.\n",
    "\n",
    "**[Note]**\n",
    "- ```langchain-opentutorial``` is a package that provides a set of easy-to-use environment setup, useful functions and utilities for tutorials. \n",
    "- You can checkout the [```langchain-opentutorial```](https://github.com/LangChain-OpenTutorial/langchain-opentutorial-pypi) for more details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture --no-stderr\n",
    "%pip install langchain-opentutorial"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install required packages\n",
    "from langchain_opentutorial import package\n",
    "\n",
    "package.install(\n",
    "    [\n",
    "        \"langsmith\",\n",
    "        \"langchain\",\n",
    "        \"langchain_core\",\n",
    "        \"langchain_community\",\n",
    "        \"langchain_openai\",\n",
    "        # \"vllm\", # this is for optional section\n",
    "    ],\n",
    "    verbose=False,\n",
    "    upgrade=False,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Environment variables have been set successfully.\n"
     ]
    }
   ],
   "source": [
    "# Set environment variables\n",
    "from langchain_opentutorial import set_env\n",
    "\n",
    "set_env(\n",
    "    {\n",
    "        \"OPENAI_API_KEY\": \"You OpenAI API KEY\",\n",
    "        \"LANGCHAIN_API_KEY\": \"LangChain API KEY\",\n",
    "        \"LANGCHAIN_TRACING_V2\": \"true\",\n",
    "        \"LANGCHAIN_ENDPOINT\": \"https://api.smith.langchain.com\",\n",
    "        \"LANGCHAIN_PROJECT\": \"Caching\",\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Alternatively, one can set environmental variables with load_dotenv\n",
    "from dotenv import load_dotenv\n",
    "\n",
    "\n",
    "load_dotenv(override=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_openai import ChatOpenAI\n",
    "from langchain_core.prompts import PromptTemplate\n",
    "\n",
    "# Create model\n",
    "llm = ChatOpenAI(model_name=\"gpt-4o-mini\")\n",
    "\n",
    "# Generate prompt\n",
    "prompt = PromptTemplate.from_template(\n",
    "    \"Sumarize about the {country} in about 200 characters\"\n",
    ")\n",
    "\n",
    "# Create chain\n",
    "chain = prompt | llm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "South Korea, located on the Korean Peninsula, is known for its rich culture, advanced technology, and vibrant economy. It features bustling cities like Seoul, renowned cuisine, and historic landmarks.\n",
      "CPU times: total: 93.8 ms\n",
      "Wall time: 1.54 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# Invoke chain\n",
    "response = chain.invoke({\"country\": \"South Korea\"})\n",
    "print(response.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ```InMemoryCache```\n",
    "First, cache the answer to the same question using ```InMemoryCache```."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.globals import set_llm_cache\n",
    "from langchain_core.caches import InMemoryCache\n",
    "\n",
    "# Set InMemoryCache\n",
    "set_llm_cache(InMemoryCache())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "South Korea is a technologically advanced country known for its fast-paced lifestyle, vibrant culture, and delicious cuisine. It is a leader in industries such as electronics, automotive, and entertainment. The country also has a rich history and beautiful landscapes, making it a popular destination for tourists.\n",
      "CPU times: total: 0 ns\n",
      "Wall time: 996 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# Invoke chain\n",
    "response = chain.invoke({\"country\": \"South Korea\"})\n",
    "print(response.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we invoke the chain with the same question."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "South Korea is a technologically advanced country known for its fast-paced lifestyle, vibrant culture, and delicious cuisine. It is a leader in industries such as electronics, automotive, and entertainment. The country also has a rich history and beautiful landscapes, making it a popular destination for tourists.\n",
      "CPU times: total: 0 ns\n",
      "Wall time: 3 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# Invoke chain\n",
    "response = chain.invoke({\"country\": \"South Korea\"})\n",
    "print(response.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that if we set ```InMemoryCache``` again, the cache will be lost and the wall time will increase."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "set_llm_cache(InMemoryCache())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "South Korea is a tech-savvy, modern country known for its vibrant culture, delicious cuisine, and booming economy. It is a highly developed nation with advanced infrastructure, high standards of living, and a strong emphasis on education. The country also has a rich history and is famous for its K-pop music and entertainment industry.\n",
      "CPU times: total: 0 ns\n",
      "Wall time: 972 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# Invoke chain\n",
    "response = chain.invoke({\"country\": \"South Korea\"})\n",
    "print(response.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ```SQLiteCache```\n",
    "Now, we cache the answer to the same question by using ```SQLiteCache```."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.cache import SQLiteCache\n",
    "from langchain_core.globals import set_llm_cache\n",
    "import os\n",
    "\n",
    "# Create cache directory\n",
    "if not os.path.exists(\"cache\"):\n",
    "    os.makedirs(\"cache\")\n",
    "\n",
    "# Set SQLiteCache\n",
    "set_llm_cache(SQLiteCache(database_path=\"cache/llm_cache.db\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "South Korea is a technologically advanced country in East Asia, known for its booming economy, vibrant pop culture, and rich history. It is home to K-pop, Samsung, and delicious cuisine like kimchi. The country also faces tensions with North Korea and strives for reunification.\n",
      "CPU times: total: 31.2 ms\n",
      "Wall time: 953 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# Invoke chain\n",
    "response = chain.invoke({\"country\": \"South Korea\"})\n",
    "print(response.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we invoke the chain with the same question."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "South Korea is a technologically advanced country in East Asia, known for its booming economy, vibrant pop culture, and rich history. It is home to K-pop, Samsung, and delicious cuisine like kimchi. The country also faces tensions with North Korea and strives for reunification.\n",
      "CPU times: total: 375 ms\n",
      "Wall time: 375 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# Invoke chain\n",
    "response = chain.invoke({\"country\": \"South Korea\"})\n",
    "print(response.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that if we use ```SQLiteCache```, setting caching again does not delete stored cache."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "set_llm_cache(SQLiteCache(database_path=\"cache/llm_cache.db\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "South Korea is a technologically advanced country in East Asia, known for its booming economy, vibrant pop culture, and rich history. It is home to K-pop, Samsung, and delicious cuisine like kimchi. The country also faces tensions with North Korea and strives for reunification.\n",
      "CPU times: total: 0 ns\n",
      "Wall time: 4.01 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# Invoke chain\n",
    "response = chain.invoke({\"country\": \"South Korea\"})\n",
    "print(response.content)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "cp311",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
